Causal LM tokenization: Chunking and seq2seq Forward - #206

This PR is the nth rewrite of causal language modeling tokenization, now changing it to:

• Let us use sequence type tokenization with causal language models if we want to
• Or enable chunking on the concatenated sequence, optionally dropping the remainder if the chunks are uneven

Sequence type tokenization is a port of the old causal LM tokenization logic with a bug fix for the concatenation sequence length; I have verified the quality matches approximately what we would expect from prompt tuning on RTE.

Chunking seems to be functional and has tests verifying output correctness but is not publicly exposed yet and therefore has not fully quality tested.

We need to be careful about collator compatibility when changing the chunking strategy, since prompt tuning and the base model resources/text generation currently manage collators for different model types differently. The main thing to be wary of is that the causal language modeling collator should not be used with the sequence approach to avoid clobbering the labels.

Note that currently things are still being statically padded and using the default collator; it would be more ideal to:

• use the padding collator for dynamic padding when possible
• add better collator based tests operating on transformers datasets; this PR does start these tests a bit, but only covers the default collator, which does not necessarily need a torch dataset to work